Introduction

A detailed summary statistics that provides the basis for the numbers in the paper. This analysis ensures the reproducibility of the pipeline and creates the statistics from raw data.

Software and packages

R version and package dependencies that were used R version 3.3.3 (2017-03-06) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.2 LTS

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=nl_NL.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=nl_NL.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=nl_NL.UTF-8 LC_NAME=nl_NL.UTF-8 LC_ADDRESS=nl_NL.UTF-8 LC_TELEPHONE=nl_NL.UTF-8 LC_MEASUREMENT=nl_NL.UTF-8 LC_IDENTIFICATION=nl_NL.UTF-8

attached base packages: [1] grid stats graphics grDevices utils datasets methods base

other attached packages: [1] markdown_0.7.7 cowplot_0.7.0 NMF_0.20.6 cluster_2.0.6 rngtools_1.2.4 pkgmaker_0.22 registry_0.3 heatmap3_1.1.1 sm_2.2-5.4 corrplot_0.77 prettyR_2.2 reshape_0.8.6 XLConnect_0.2-12 XLConnectJars_0.2-12 readr_1.1.0 qdap_2.2.5 qdapTools_1.3.1 qdapRegex_0.7.2 qdapDictionaries_1.0.6 gridExtra_2.2.1 reshape2_1.4.2 stringr_1.2.0 mvtnorm_1.0-6 plyr_1.8.4 RColorBrewer_1.1-2
[26] ggplot2_2.2.1 drc_3.0-1 MASS_7.3-45

loaded via a namespace (and not attached): [1] nlme_3.1-131 bitops_1.0-6 pbkrtest_0.4-7 doParallel_1.0.10 tools_3.3.3 R6_2.2.0 DBI_0.6-1 lazyeval_0.2.0 mgcv_1.8-16 colorspace_1.3-2 openNLPdata_1.5.3-2 nnet_7.3-12 chron_2.3-50 quantreg_5.29 reports_0.1.4 SparseM_1.76 NLP_0.1-10 sandwich_2.3-4 labeling_0.3 slam_0.1-40 scales_0.4.1 tm_0.7-1 digest_0.6.12 minqa_1.2.4 lme4_1.1-12 plotrix_3.6-4 zoo_1.7-14 openNLP_0.2-6 gtools_3.5.0
[30] dplyr_0.5.0 xlsx_0.5.7 car_2.1-4 RCurl_1.95-4.8 magrittr_1.5 wordcloud_2.5 Matrix_1.2-8 Rcpp_0.12.10 munsell_0.4.3 stringi_1.1.5 multcomp_1.4-6 parallel_3.3.3 gdata_2.17.0 gender_0.5.1 lattice_0.20-35 splines_3.3.3 xlsxjars_0.6.1 hms_0.3 venneuler_1.1-0 igraph_1.0.1 fastcluster_1.1.22 codetools_0.2-15 XML_3.98-1.6 data.table_1.10.4 nloptr_1.0.4 foreach_1.4.3 MatrixModels_0.4-1 gtable_0.2.0 assertthat_0.1
[59] gridBase_0.4-7 xtable_1.8-2 survival_2.41-3 tibble_1.3.0 rJava_0.9-8 iterators_1.0.8 TH.data_1.0-8

Data description

Number of unique strains: 124

Table containing the number of strains and antimicrobials

  Azithromycin Cefixime Ceftriaxone Ciprofloxacin Penicillin Spectinomycin Tetracycline Sum
training 84 84 84 84 84 84 84 588
validation 40 40 40 40 40 40 40 280
Sum 124 124 124 124 124 124 124 868

Dose response modelling

Samples that are above or below limit of detection new method (split according to antimicrobials)

  Azithromycin Cefixime Ceftriaxone Ciprofloxacin Penicillin Spectinomycin Tetracycline Sum
above limit of detection 4 0 0 0 7 4 0 15
quality ok 120 124 124 124 117 120 124 853
Sum 124 124 124 124 124 124 124 868

Limit of detection (excluding data with Etest above limit of detection, if dose response does not result in limit of detection)

  training validation Sum
above limit of detection 6 9 15
quality ok 582 271 853
Sum 588 280 868

Summary of samples that are above limit of detection new method (all samples summarized, limit of detection includes also Etest limit of detection)

  training validation Sum
limit of detection 17 14 31
quality ok 571 266 837
Sum 588 280 868

Dose response curves for all antimicrobials (training and validation data):

Figure 1. Potency shift of antimicrobials across different strains of N. gonorrhoeae. Dose response curves for all strains and antimicrobials are shown (except samples above limit of detection). Strains that were classified as susceptible according to EUCAST 2016 MIC breakpoints were coloured in green, intermediate resistant strains in blue and resistant strains in red. The gradual shift of the potencies (EC50) towards higher concentrations can be observed for all antimicrobials.

Regression analysis

## [[1]]
## [[1]]$Estimates
## 
## Call:
## lm(formula = log(etest) ~ (esti))
## 
## Coefficients:
## (Intercept)         esti  
##       1.101        1.001  
## 
## 
## [[1]]$Matrix
## [[1]]$Matrix[[1]]
##              (Intercept)         esti
## (Intercept) 0.0023234883 0.0003695523
## esti        0.0003695523 0.0002613105
## 
## [[1]]$Matrix$Summary
## 
## Call:
## lm(formula = log(etest) ~ (esti))
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -4.1999 -0.6237  0.0616  0.7230  3.0847 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)    
## (Intercept)  1.10145    0.04820   22.85   <2e-16 ***
## esti         1.00057    0.01617   61.90   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 1.014 on 569 degrees of freedom
## Multiple R-squared:  0.8707, Adjusted R-squared:  0.8705 
## F-statistic:  3831 on 1 and 569 DF,  p-value: < 2.2e-16

Pearson’s correlation coefficient

[1] 0.9331071

Outlier

Outliers were defined as higher than +/- four doubling dilutions different from Etest MIC. The column “deviation” displays doubling dilutions deviation of predicted values from MIC. The column “compare” displays the EUCAST to the predicted classification.

ID strain antibiotic MIC Etest_predicted deviation compare
11_Ceftriaxone_40strains2.txt Fluorometric11 Ceftriaxone 0.002 0.4702 7.877 S_to_R
14_Cefixime_40strains2.txt Fluorometric14 Cefixime 0.016 1.083 6.081 S_to_R
17_Ceftriaxone_40strains2.txt Fluorometric17 Ceftriaxone 0.004 2.387 9.221 S_to_R
18_Cefixime_40strains2.txt Fluorometric18 Cefixime 0.016 9.451 9.206 S_to_R
18_Ceftriaxone_40strains2.txt Fluorometric18 Ceftriaxone 0.002 0.4898 7.936 S_to_R
18_Penicillin_80strains3.txt Fluorometric18 Penicillin 32 1.923 -4.057 R_to_R
1_Ceftriaxone_40strains1.txt Fluorometric1 Ceftriaxone 0.002 0.3237 7.339 S_to_R
33_Ceftriaxone_80strains5.txt Fluorometric33 Ceftriaxone 0.004 0.1736 5.44 S_to_R
34_Ceftriaxone_80strains5.txt Fluorometric34 Ceftriaxone 0.004 0.06407 4.002 S_to_S
37_Ceftriaxone_80strains5.txt Fluorometric37 Ceftriaxone 0.008 0.2569 5.005 S_to_R
4_Penicillin_40strains1.txt Fluorometric4 Penicillin 3 118.8 5.307 R_to_R
4_Penicillin_80strains1.txt Fluorometric4 Penicillin 4 78.77 4.3 R_to_R
57_Cefixime_80strains11.txt Fluorometric57 Cefixime 0.016 0.3266 4.352 S_to_R
5_Penicillin_40strains1.txt Fluorometric5 Penicillin 3 91.36 4.929 R_to_R
5_Spectinomycin_40strains1.txt Fluorometric5 Spectinomycin 1024 25974 4.665 R_to_R
60_Cefixime_80strains8.txt Fluorometric60 Cefixime 0.016 1.067 6.059 S_to_R
66_Ceftriaxone_80strains9.txt Fluorometric66 Ceftriaxone 0.004 0.000183 -4.45 S_to_S
73_Cefixime_80strains9.txt Fluorometric73 Cefixime 0.016 0.2623 4.035 S_to_R
73_Ceftriaxone_80strains9.txt Fluorometric73 Ceftriaxone 0.008 0.1709 4.417 S_to_R

Results from the regression analysis show that the correlation with Etest is excellent however all values are systematically shifted towards lower values.

Figure 2. Correlation and deviations between the Etest MICs and predicted MICs. (a) Linear regression between EC50 and Etest MIC for the training data (84 strains). The Pearson’s correlation coefficient for the linear regression (blue line) was 0.93 and the confidence interval highlighted in grey. Slope and intercept for a perfect correlation was drawn as dashed black line for comparison. (b) The kernel density function of the EC50 values for the training data (n=269) is shown in red (median -1.68). The kernel density of the predicted MICs for training and validation data (n=837) is shown in purple (median -0.015). (c) Deviations of predicted MICs from Etest MIC per antimicrobial (n=837). The boxplots show the median and 25%-75% quartiles. The whiskers span the range from the bottom 5% to the highest 95% of the data. The essential agreement (EA) is written below the boxplots.

Categorical agreement with Etest

Table containing the categories (assuming Etest MIC according to EUCAST categories as gold standard)

  Azithromycin Cefixime Ceftriaxone Ciprofloxacin Penicillin Spectinomycin Tetracycline Sum
I 38 0 0 0 37 0 25 100
R 60 35 10 87 82 7 96 377
S 26 89 114 37 5 117 3 391
Sum 124 124 124 124 124 124 124 868

Categorical agreement, absolute and in percentage

EUCAST as rows, predicted categories as columns

  S I R Sum
S 307 8 76 391
I 13 42 45 100
R 1 12 364 377
Sum 321 62 485 868
  S I R Sum
S 0.354 0.009 0.088 0.45
I 0.015 0.048 0.052 0.115
R 0.001 0.014 0.419 0.434
Sum 0.37 0.071 0.559 1

Categorical agreement, by antibiotics, absolute and in percentage

EUCAST_to_predicted_categories

  Azithromycin Cefixime Ceftriaxone Ciprofloxacin Penicillin Spectinomycin Tetracycline Sum
I_to_I 9 0 0 0 30 0 3 42
I_to_R 17 0 0 0 7 0 21 45
I_to_S 12 0 0 0 0 0 1 13
R_to_I 6 0 0 0 6 0 0 12
R_to_R 54 35 9 87 76 7 96 364
R_to_S 0 0 1 0 0 0 0 1
S_to_I 5 0 0 1 1 0 1 8
S_to_R 5 29 40 0 0 0 2 76
S_to_S 16 60 74 36 4 117 0 307
Sum 124 124 124 124 124 124 124 868
  Azithromycin Cefixime Ceftriaxone Ciprofloxacin Penicillin Spectinomycin Tetracycline Sum
I_to_I 0.01 0 0 0 0.035 0 0.003 0.048
I_to_R 0.02 0 0 0 0.008 0 0.024 0.052
I_to_S 0.014 0 0 0 0 0 0.001 0.015
R_to_I 0.007 0 0 0 0.007 0 0 0.014
R_to_R 0.062 0.04 0.01 0.1 0.088 0.008 0.111 0.419
R_to_S 0 0 0.001 0 0 0 0 0.001
S_to_I 0.006 0 0 0.001 0.001 0 0.001 0.009
S_to_R 0.006 0.033 0.046 0 0 0 0.002 0.088
S_to_S 0.018 0.069 0.085 0.041 0.005 0.135 0 0.354
Sum 0.143 0.143 0.143 0.143 0.143 0.143 0.143 1

CI spanning over two categories (without limit of detection)

  Azithromycin Cefixime Ceftriaxone Ciprofloxacin Penicillin Spectinomycin Tetracycline Sum
0 91 93 88 114 87 110 116 699
1 29 19 36 6 30 10 8 138
Sum 120 112 124 120 117 120 124 837

Figure 3. Contingency table with categorical errors of model predicted MICs. Etest MIC data were classified into the categories resistant (R), susceptible (S) and intermediate resistant (I) according to the EUCAST 2016 criteria. The cutoff values (mg/L) are shown as dashed black lines. Predicted MIC values (n=868) are shown as point estimates (black dots) with 95% confidence interval (colored dashes). For some estimates no confidence interval could be calculated (limit of detection), those were drawn as triangles. Correctly classified strains are drawn in green. Minor errors resulting from misclassifications of intermediate strains are drawn in blue. Major errors (S to R) were found for ceftriaxone (n=40), cefixime (n=29), azithromycin (n=5) and tetracycline (n=2). One very major error (R to S) was found for ceftriaxone (red). A high number of estimates (n=140) has confidence intervals spanning two categories.

Specificity and sensitivity

Note: I counted as R

Categorical agreement summary

EUCAST as rows, predicted categories as columns

  R S
R 463 14
S 84 307
R_to_R R_to_S S_to_R S_to_S
463 14 84 307

Sensitivity

\(\frac{R-to-R}{R-to-R + R-to-S} =\) 0.9706

### 95% CI 
binom.test(463, 477, p = 0.5,conf.level = 0.95)
## 
##  Exact binomial test
## 
## data:  463 and 477
## number of successes = 463, number of trials = 477, p-value < 2.2e-16
## alternative hypothesis: true probability of success is not equal to 0.5
## 95 percent confidence interval:
##  0.9512455 0.9838631
## sample estimates:
## probability of success 
##              0.9706499

Specificity

\(\frac{S-to-S}{S-to-S + S-to-R} =\) 0.7852

### 95% CI 
binom.test(307, 389, p = 0.5,conf.level = 0.95)
## 
##  Exact binomial test
## 
## data:  307 and 389
## number of successes = 307, number of trials = 389, p-value < 2.2e-16
## alternative hypothesis: true probability of success is not equal to 0.5
## 95 percent confidence interval:
##  0.7452549 0.8286867
## sample estimates:
## probability of success 
##              0.7892031

Essential agreement with Etest

Etest_deviation AZM CFM CRO CIP PEN SPT TET
onedeviation 53.3 30.4 52.4 57.5 60.7 57.5 58.1
twodeviation 95 67 79 93.3 82.9 89.2 93.5
fourdeviation 100 95.5 92.7 100 96.6 99.2 100

Supplementary Material

Timecourse

8 WHO reference strains were followed over a time-course of 0-15 hours and measured every 3 hours (one replicate)

Figure S1. Fluorescence based time-kill curves. Logarithmized fluorescence values are plotted against the time (h). Ten different dilutions of each antimicrobial, positive control (Inf) and negative control (conc. 0) were tested on eight WHO reference panel strains. Start concentrations were calibrated to approximately 107 CFU/ml which corresponds to a log fluorescence of 6. From 0-3 hours negative controls without antimicrobial resulted in decreased bacterial numbers, at 6 hours all samples show increased fluorescence.

Coefficient of variation

8 Reference strains were used in this analysis. 3 replicates with seven antimicrobials (n=56) were used to calculate the coefficient of variation

strain antibiotic MIC mean sd CV
WHO F Azithromycin 0.125 0.01394 0.009803 0.7031
WHO F Cefixime <0.016 0.0005485 4.743e-05 0.08647
WHO F Ceftriaxone <0.002 0.0002095 0.0001515 0.7231
WHO F Ciprofloxacin 0.004 0.002442 0.0003445 0.141
WHO F Penicillin 0.032 0.005978 0.0005169 0.08648
WHO F Spectinomycin 16 3.141 2.253 0.7174
WHO F Tetracycline 0.25 0.04333 0.02345 0.5413
WHO G Azithromycin 0.25 0.03174 0.0008395 0.02645
WHO G Cefixime <0.016 0.003609 6.18e-05 0.01712
WHO G Ceftriaxone 0.008 0.00183 0.000874 0.4776
WHO G Ciprofloxacin 0.125 0.04593 0.003962 0.08626
WHO G Penicillin 0.5 0.1495 0.03925 0.2625
WHO G Spectinomycin 16 1.556 1.256 0.807
WHO G Tetracycline 32 9.979 2.121 0.2126
WHO K Azithromycin 0.25 0.04507 0.005532 0.1227
WHO K Cefixime 0.25 0.1223 0.01594 0.1303
WHO K Ceftriaxone 0.064 0.02189 0.01038 0.474
WHO K Ciprofloxacin >32 9.845 1.578 0.1603
WHO K Penicillin 2 0.7465 0.07489 0.1003
WHO K Spectinomycin 16 2.669 1.447 0.542
WHO K Tetracycline 2 1.368 0.1663 0.1215
WHO L Azithromycin 0.5 0.04175 0.007214 0.1728
WHO L Cefixime 0.125 0.06644 0.007194 0.1083
WHO L Ceftriaxone 0.25 0.05039 0.02726 0.5409
WHO L Ciprofloxacin >32 4.47 0.626 0.1401
WHO L Penicillin 2 0.9015 0.0676 0.07498
WHO L Spectinomycin 16 1.919 1.076 0.5605
WHO L Tetracycline 2 1.187 0.2155 0.1815
WHO M Azithromycin 0.25 0.0404 0.003365 0.08328
WHO M Cefixime <0.016 0.00291 0.0003033 0.1042
WHO M Ceftriaxone 0.012 0.001849 0.0008431 0.4558
WHO M Ciprofloxacin 2 0.2936 0.03352 0.1142
WHO M Penicillin 8 22.13 1.932 0.08727
WHO M Spectinomycin 16 2.266 1.16 0.5119
WHO M Tetracycline 2 0.9742 0.09473 0.09724
WHO N Azithromycin 0.25 0.02093 0.00204 0.0975
WHO N Cefixime <0.016 0.004648 0.0003874 0.08336
WHO N Ceftriaxone 0.004 0.001196 0.0007731 0.6463
WHO N Ciprofloxacin 4 1.588 0.03178 0.02001
WHO N Penicillin 8 2.304 1.996 0.8663
WHO N Spectinomycin 16 1.162 0.8212 0.7065
WHO N Tetracycline 16 6.36 0.4289 0.06742
WHO O Azithromycin 0.25 0.04498 0.007163 0.1592
WHO O Cefixime 0.016 0.009882 0.004161 0.4211
WHO O Ceftriaxone 0.032 0.004744 0.003545 0.7473
WHO O Ciprofloxacin 0.008 0.002171 0.0003707 0.1707
WHO O Penicillin >32 10.29 4.058 0.3944
WHO O Spectinomycin >1024 462.4 167.8 0.3629
WHO O Tetracycline 2 0.9739 0.1611 0.1655
WHO P Azithromycin 4 0.2521 0.006027 0.0239
WHO P Cefixime <0.016 0.003542 0.0006906 0.195
WHO P Ceftriaxone 0.004 0.001126 0.0005669 0.5037
WHO P Ciprofloxacin 0.004 0.001845 0.0001956 0.106
WHO P Penicillin 0.25 0.09521 0.001573 0.01652
WHO P Spectinomycin 8 2.168 1.272 0.5867
WHO P Tetracycline 1 0.4477 0.1042 0.2327 
print(summary(cv$CV))
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
## 0.01652 0.09743 0.16810 0.29190 0.50570 0.86630

Figure S2. Intra assay coefficient of variation. To test the reproducibility of the resazurin MIC assay seven antimicrobials were tested on eight WHO reference strains (n=56). The mean and standard deviation of three independent experiments was calculated. The coefficient of variation (ratio of standard deviation over the mean) was calculated for sample. Barplots are shown for each sample. The mean of the coefficient of variation (intra assay CV) is 0.29.

Hill coefficient statistics

It was tested if the Hill coefficient was significantly different (t-test) between antimicrobials and strains.

  antibiotic mean sd min max
3 Ceftriaxone 1.6 1.3 0.1124 7.455
2 Cefixime 1.9 1.5 0.1095 7.151
7 Tetracycline 2.1 0.9 0.4026 7.818
5 Penicillin 2.5 1.7 0.3353 10.47
1 Azithromycin 2.6 1.5 0.6032 13.44
4 Ciprofloxacin 2.7 1.2 0.2596 7.733
6 Spectinomycin 2.9 1.7 0.3138 9.158

Figure S3. Difference of Hill coefficients. (a) The difference between the mean of 124 Hill coefficients (124 clinical strains examined) is shown for each antimicrobial combination. High values are shown in an increasingly intense blue colour gradient and low values in red. A pairwise t-test was performed and non-significant differences (p value > 0.05) marked with a black cross. (b) Hierarchical clustering of Hill coefficients. Rows represent Hill coefficients for different strains (N=124) and columns antimicrobials. The beta-lactams penicillin G, ceftriaxone and cefixime are more similar to each other than to the other antimicrobials.

Example for Biphasic curves

The dose response curves with four parameters might not capture the effects for beta-lactams well. To demonstrate this one example is shown here in detail. A biphasic model fits this data better than the simpler four parameter function. However to fit this model without overparametrization, this requires dense dose spacing and at least three replicates per sample.

Figure S4. Biphasic dose response curves. The viability (%) was plotted against 24 different antimicrobial concentrations. Mean and standard error of three independent experiments are shown. (a) Ceftriaxone in Strain 11 (validation data). A biphasic model (red curve) fits the model better (bic=563) than a monophasic model (bic=794).1 The first EC50 is at 0.12 mg/L and the second at 1.21 mg/L (Etest MIC=0.125 mg/L). (b) Cefixime in Strain 11 (validation data). A biphasic model (red curve) fits the model better (bic=850) than a monophasic model (bic=8574). The first EC50 is at 0.16 mg/L and the second at 1.39 mg/L (Etest MIC=0.25 mg/L).